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SUMMARY 



Problem 

The optimum weighting of variables to predict a dependent criterion 
variable is an important problem in nearly all of the social and natural 
sciences. Although the predominant method ^ multiple regression analysis, 
yields optimum weights for the sample at hand, these weights are not 
generally optimum in the population from which the sample was drawn* 
However* all o ther multivariate methods proposed to date have provided 
poorer prediction in some samples and populations, 

Obj ec tive 

The objective was to develop and validate a multivariate method that 
would provide better prediction weights than existing methods. The weights 
would be more stable from sample to sample and would predict the dependent/ 
criterion variable more accurately in the population* 

Approach 

A method was developed that sacrifices some "prediction 11 in the sample 
at hand in order to achieve a more reliable and stable predictor composite. 
Multiple regression (HR) does the opposite^ assuming completely reliable 
and trustworthy data* The method developed. Factor Regression Analysis (FRA) , 
is based on the first principle component of the predictor; intercorrelation 
matrix with validities in the diagonal cells* 

Two kinds of data were used to evaluate FRA: Navy data and Monte Carlo 
data, Hundreds of samples were analyzed * and the welghti were applied to 
the populations from which the samples were drawn* All predictor subset 
sizes were analysed for each sample* 

Findings 

The new method, FRA* yielded very stable predictor composites and weights- 
the weights themselves varied less from sample to sample than did MR weights 
from the same samples, These differences were marked for low sample sizes 
(e.g., N * 25), regardless of the number of variables in regression* For larg 
sample sizes (e.g*, N - 200), the differences were smaller hut still fairly 
consistent. 

For small samples, FRA. composites were much more valid in the population 
than the MR composites based on the same samples. The number of predictors 
in the subset did not turn out to be very important, FRA weights based on 
samples of 25 were about as valid as MR weights based on samples of 100. With 
samples of 200 the two methods yielded roughly equivalent prediction, 
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Cgnc 1 u s t a n s 

The new method, FRA ? Is a very important improvement upon multiple regres- 
sion fox small samples. In additions FRA does not fail at the higher samples 
sizms previously proposed methods have done. Thus, it can be used with 
confidence for all samples sizes as they occur in applied settings* 
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INTRODUCTION 



Frobl am 

Since 1900, it has been commonly assumed that multiple-regression 
weights based on a sample are the best candidates for prediction weights 
in the population (see Gullikson, 1950, & Walker, 1958), Evidence that 
this is generally not true for small samples has been accumulating for 
many years (e.g. , Horst, 1941; Guilford, 1954; Schmidt, 1971), Although 
several multivariate methods have bien advocated in recognition o$ this 
fact (Big, j Horst, 1941; Kendall, Note 1; Rock, Linn, Evans, & Patrick, 
1970) , none have consistently excelled multiple regression, even with 
rather small samples (e,g,, N = 50) > In fact, a recent study (Rock et al, , 
1970) found that sample multiple-regression weights predicted Y to the 
population better than did weights computed by the other promising multi- 
variate methods, 

Multiple regression yields a predictor composite that is s for the 
sample at hand, maximally related to the criterion. However % in the pro-* 
cess, multiple regression violates a principle for constructing a reliable 
composite: the higher the Intercorrelatiori among the components of the 
composite, the higher the reliability of the composite (Moiser, 1943) , 
When multiple regression computes the optimum weights! it computer the 
weight for each predictor while all other predictors are held constant* 
Thus, each predictor's weight is baited on its validity after all the 
criterion-related variance that the predictor shares with the othar pre- 
dictors has been removed. In effect* eacli weight is based on a residual 
matrix or, equivaiently , partial correlation* When two predictors; are 
highly correlated, the one with the larger validity will tend to receive 
a representative weight and the othw a much lower, or even negative 
weight, There is no overlap between any of the weights as far as account- 
ing for criterion variance is concerned* Multiple-regression rul^s out any 
capitalization upon the redundancy that invariably exists among the pre- 
dictors. In contrast, reliable composites tend to give great weight to 
intercorrelation among the predictors* 

This inconsistency between reliability o£ the predictor composite, and 
optimum rf prediction lf in the sample at hand, can be seen in the formula for 
the standard error of multiple-regression weights. Examination of that 
formula leads to the unmistakable conclusion that the higher the correlations 
among the predictors, the higher the standard errors of the weights* this 
indicates an unreliable composite irt a situation that should yield a very 
reliable composite. Beaton, Rubin,, and Bar one (1970) demonstrated this 
empirically, showing that the higher the intercorrelation, the more unstable 
the multiple-regression weights. 
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Another inadequacy in multiple regression ims been referred to as Che 
"par Ci ailing fallacy" (Gordon, 1968), As mentioned above, multiple regression 
is based on par t iai^corteiat ional techniques,. It determines optimum weights 
by par tialling— each predictor 1 s optimum weight is determined by par tial ling 
out association with th% other predictors, Unfortunately, partial correlations 
are based on the assumptions that the variable feeing partialled out contains 
no unique component and is measured without error (Burks, 1926; Monroe & 
Stulc, 1935; Stephenson,, 1935; Gordon, 1968), FW, if any, psychological 
tests and measurements satisfy both of these assumptions. Unf or tunately , 
no publications could fa§ found that demonstrated empirically how the fal- 
lacious partial correlations affected, or did W§ affect, multiple-regres- 
sion weights or their validity. 

It would appear* therefore, from several standpoints, that multiple 
regression does not glva due consideration to zhM unreliability and complexity 
of the measures used throughout most of the behavioral and social sciences* 
Perhaps compromise between composite reliability and maximization of composite 
validity in the sample ax hand , would yield better weights for prediction in 
the population. 

Purpose 

The purpose of this effort was to develop and validate a multivariate 
method that would provide better predictor; weights than existing methods, 
These weights would be more stable from sample to sample and would predict 
the dependent/criterion variable more accurately in the population. 
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PROCEDURE 



Factor Regression Analysis {FRA) 

The method developed is called Factor Regression Analysis (FRA) * Only 
the empirical steps in the actual procedure will be presented here, A 
presentation of FRA and its relationship to multiple regression, principle 
comp^ ients 5 and factor analysis will be found in the discussion section. 

Starting with the intsrcorrelatlon matrix of predictors* the unities 
in the main diagonal cells were replaced by validities— each predictor's 
validity was placed in thm predictor's diagonal cell. The first principle 
component of this matrix was computed (Gulliksen, 1950, p, 346) , and the 
loadings of the predictora On the first principle component were used as 
predictor weights. 

Predictors were droppad in a stepwise manner by omitting the predict 
tor with the smallest loading, The dropped predictor was omitted from the 
correlation matrix, and a new principle component was computed. This pro- 
cedure was followed until only two predictors remained, 

Multiple Refir as sio n 

Step-wise multiple regression was used* since it was considered pre- 
ferable to the "forward" selection and the "backward 11 elimination procedure 
(Kerlinger & Pedhagur, 1973), At each step, all possible combinations, of 
particular size, were compared- The selected subset was the one that cor- 
related highest with Y, 

Test Data 

Two kinds of data were used to compare FRA and multiple regression (MR) 
Navy data and Monte Carlo data, The Navy data consisted of 16*826 records 
for enlisted men, obtained item the Naval Examination Center, A composite 
variable, Final Multiple Score (FM5) * which is used for the advancement 
(promotion) decision, was the criterion variable (Robertson, JameSj & 
Royle^ 1972) - Ten predictor© were used- eight aptitude tests (the Navy 
Basic Test Battery) s age, and amount. of education (in years), The 16,826 
records were placed in random order, and successive groups were used as 
samples. All of the records were used as the population, 

Monte Carlo data were generated from published correlation matrices 
(Rock et al«, 1970; McCorn^akf, 1370), Predictors ranged in number from 11 
to 15, A computer program (Mopnan & Cohver, 1973, Note 2) was used to gene 
Monte Carlo samples based on the published correlation matrices. 

Population Validity 

The weights for each pra41ctor set or subset were applied to the popula 
tipn through the use of the following formulas 
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where! 

* sample weight for cho i-th predictor, 

r ±¥ " P°P u lation validity of the 1-th predictor , 

* 0p/o s — the standard deviation in the populatiotij over the standard 
deviation in the sample, and 

r^j = population correlation hetween the i^th aiid j^th predictor « . 

This is called the "population validity, 11 It indicates liow^eil the sarnple 
weights would work in the population, FBA and MR fights were evaluated 
using this formula. 

Computer Program 

-4 computer program was developed for the GA1830 Comptiter to perform the 
calculations , 
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RESULTS 



The inter correla tion matrix for the "population" of Navy data (N_ « 16,826) 
is presented in Tabic 1* Multlple-rigr#§s ioti weights for these data wires 

X Variable 12345 6789 10 

MR Weight i .15 ,08 ,12 -.01 . 01 -,OS .11 .11 .07 .09 

The multiple correlation was .46* In the first sample (N « 25), the weights 
werei 

MR Weight : ,10 -. 55 .49 ,21 49 ,04 ,62 -,04 ,28 -.25 

PRA Weight i ,30 ,28 ,54 ,26 ,10 ,18 .55 .34 ,08 ,09 

These weights yielded a population validity of ,24 in the case of MR, and ,42 
in the case of FRA- Sample validities fQr ttiese two sets of weights were ,75 
and *56j respectively. 



Table 1 

Intercoxr elation. Matrix fat the ^avy Data 
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Table 2 presents the average popUl^tiLOti ^Jtetlty itf 1,0 g§m$lm$* TMm 
is given separately for four sample (25, SOi lOQ * a.nd lO-O) s w*elx 

as for predictor subsets of all pos^ibii sigis # Of m^t Iftt^r^se id tie 
table is the "Mean Differatice n Qoluton. £t &hom the ddff et^ne^, on tl>e 
average s in the population validitiei Joit KB aifltl FRJ^ v^igik^ g:iv@n J.a -the? 
first two columns* Each value fefera tfQ a dlf^Wiiit p#^d ic^Qr susbg#t ^t^e 
from 2 to 10, It can be seen that the ^EtA v/eights vet*£ mAhi miPte? v^li^ 
than the >1R weights. Except fis? ve^y giftail Suji&ts p > CTo or' tinted 

it mattered little how- many pr^4i.ctc?ts r e in sublet * 

The mean differences ware *iUijS la^Se 1t3 mmy instances, e^g, > J5 ^ 
so correlation points, Even with, Qi^-ly f o ut pr Victors tn tW $^h<$tt aui4 
an N of 50, the mean difference w^§ li^ In getf^r U , -tf&tghtfs based On, 
samples of 25 were as valid as ftR, ^elggita Wsed oft mnplm^ e*f 3-0O- 

For the largest samples Qj * 2QG?) ? HR Veigfcits weFe t itctl# b etcir f g r 
the small subsets of tvo or thM& Pr^dd^t However ^ FJ4 Wedgh-ts acfite^ed 

an average validity of -43 witli six pr#cli cta?rs^ a level n^Vest #chir^ed by 
MR weights for any subset* 

Standard deviations of the Population VaiUdlc ics a^fo^s lO samples ire 
presented in the last two columns of T^l-e 2. 4s can fc>6 ££e?n, the popula- 
tion validities differed greatly for MB b^t notZ f^t FRA t« f^ct 1 the^ 
were nearly ai consistent, in the £a,ie Q £ t&A, | ^ 2 J when l£ = ^OO. 

This was definitely not tfu© for 3ft. 

The standard deviations of tfos predictor- tf^tglits, *ppp'°s^3 to th& 
standard deviations of the population \?aldd±,ties ? are p?te^^n.ted W "Cab fife J* 
These standard deviations pertain on ly to th±e Weights whe^ a ll Pr^edifitOTs 
were in regression* The variafril ity 0 £ mMhi pT^ddctot r i tfei.Shti a*ctc^s lQ 
samples is indicated for MR md FJA* it tfin.bg g^in tjaat SRA Veigbta 
were much more stable on the average * Ihd§ is fcsjeclally tr^e Jo^ ^all 
samples* However, even when II s JQO 1 tfhe HEt V^tgtt fo^ p^ld ictortf £outf, &txi 
arid ten varied much more than the PR^A weight * 

Before proceeding to other po julat£Oti#s the selection of p^id^-cors 
inclusion in the subsets should b# e35air3iii£<L *Cab3-e 4 ^ho^s the p*etcenctg^ 
of predictors in common for th# MB a*d m^hm^tm N * 50*, 3t is ci^ar- 

that MR and FKA did riot use the s^e predict ot^. ReauILts fo^ che otheTcr s^pl 
si^es were almost identical and tl^r^f o^e ^ net gtv^en he^e^ tf^OTple sdzm 

vslb simply not important in this iresject, l-n 1^0 case s ev#^ -foe Irf^sfi ^iinjleB 
was there complete overlap in the sm^il subs-et^. Fu.rtl»er ^ t^ere ^eri itnan^y 
caaea in whieli only one^Chird cot%io*ft, rie la^t col^Umtf sJowa |th& avartfgfe 
percent for each subset siEe . 

Three other ff populations 1 ' weteu^ed^ tc3 e^l^UaCe fRA-i ^ag dtfte aboytf 
using Navy data, There is one impottftrvt dtfie^ence^ ho*%V6r * Instead of 
using whatevex data were availebl#» ^ub*lt#heJ cL&t^ ttat h^d ^e©n l^^ed 
in journal articles were used, Ttiia en^biid ti» e cQiri^air isc?^ tff ^e^ui ts 5 a end 
the choice of widely differing popul^ti^n^* 
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37 


41 


04 


&5 


04 


7 


36 


42 " 


05 


06 


04 


1 


36 


41 


05 


OS 


03 


9 


36 


42 


06 


05 


03 


10 


36 


42 


Q6 


05 


02 





I " 


200 








I 


38 


34 


-04 


01 


04 


3 


40 


36 


-04 


02 


02 


4 


41 


4Q 




02 


01 


5 


42 


42 


00 


01 


□3 


b 


42 


43 


01 


02 


03 


1 


41 


43 


01 


02 


03 


S 


41 


41 


01 


02 


03 


9 


41 


42 


01 


02 


02 


19 


u 


42 


°1 


02 


01 
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Table 3 

Standard Deviation of Weights Across 10 Samples of Navy, Data 



Predict or 



heights 


1 


2 


J 


4 


5 


6 


7 


a 


9 


10 




— - -- - - 






1 


•- 25 












MR. 


.29 


.35 


,30 


.20 


.31 


, 23 


.21 


.21 


.25 


,29 


FRA 


.19 


. 12 


, 20 


. 15 


.05 


. 06 


.08 


. 12 


,08 


.21 










N 


= 50 












ML 


,14 


. 12 


. 14 


.07 


,06 


,09 


.12 


.15 


.15 


,13 


A 


-.06 


. lit 


. 1 J 


. lu 


. 1U 


• 09 


■ Uo 


* JL J 


• lb 












1 


= 100 












m 


,11 


, 19 


. 12 


.13 


.05 


,06 


.18 


,10 


.16 


.10 




,04 


, 06 


,04 


.05 


.08 


,08 


.06 


.07 


.15 


.06 










N 


= ZOO 












m 


,06 


, 09 


. 10 


.10 


,03 


,10 


.07 


.08 


.08 


.11 


MA 


,02 


. 05 


, 11 


.02 


.03 


.04 


.04 


.07 


.06 


.04 
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Percentage of Predictors In Common for the MR a/ftd FRA Subsets 



Number of 

Predictors Sample . ___ ^ Average 



in Regression 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


Percentage 


2 


50 


50 


50 


50 


50 


50 


00 


50 


50 


50 


45 


3 


67 


33 


33 


33 


33 


67 


00 


33 


67 


33 


38 


4 


75 


SO 


75 


75 


50 


50 


25 


25 


50 


25 


50 


5 


60 


40 


80 


60 


40 


40 


40 


20 


60 


60 


50 


6 


67 


67 


83 


67 


50 


50 


67 


33 


50 


67 


60 


? 


71 


71 


86 


71 


71 


71 


86 


57 


71 


71 


73 


8 


75 


87 


87 


87 


75 


87 


87 


75 


75 


57 


82 


9 


89 


89 


89 


89 


89 


89 


100 


89 


89 


€9 


90 


10 


100 


1O0 


100 100 


1O0 


100 


100 


100 


100 


100 


100 



Two of the populations used were analysed in a journal articLe (Rock at al* p 
1970) that compared several predictor weighting and selection techniqu.ea— in- 
cluding multiple regression. In that study, multipie^agression weights wire 
shown to Be better than weights derived by any of the ether techniques. If 
FRA weights are mora valid on these data, it would be 'very strong evidence, 
indeed, for the method r s utility. Fur thermore, the tTO populations were 
originally chosen by Rock et al. (1970) because they differed greatly in terms 
of level of . validities and level of predictor inter/correlation, 

Th<e intercerrelation matrices of the two Rock at ai» s (1970) populations 
are presented in Tables 5 and 6 P and average population validities for 10 
Monte Carlo samples are presented in Tables 7 and 8, Sy n.nd large* the 
results substantiate the results for Navy data, The F^A weights were much 
more valid for N = 25 and nearly identical to MR weight validities for H = 20G. 
As in the case of Navy data, FHA weights based on samples of 25 were aibouC as 
valid as MR weights based en samples of 100, Even wheti the subset contained 
only two predictors * FRA weights were as valid as MR weights, 

Sample-to-sample fluctuations of tjie population validities for FRA were 
very sniali, as Indicated by the standard deviations given in. the last column 
of the tables. For example, in Table -7^- when II m 25 &t\d when 12 predictors 
were in regression, the standard deviation of populati^ii validities were *01 
for FRA* compared to -15 for Ml. This is for an average validity of ,71 for 
FRA and ,52 for >R 8 Comparing this with N = 100 and 13 predictors, the 
average population validity for HE is now .71 alio, but its standard deviation 
is ,02— still not as low as the standard deviation for when ^ 25* 
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Tabl« 5 

Incercorrelation Matrix for Rock I 







X 2 


y 

X 3 


X 4 


X 5 


X 6 


, X 7 


X 8 


X 9 


X 10 


X ll 


X 12 


Y 


55 


38 


64 


61 


56 


64 


62 


61 


40 


41 


45 


43 


x i 


— - 


58 


66 


66 


66 


64 


65 


73 


49 


45 


46 


46 


X 2 




— 


70 


66 


64 


65 


68 


57 


45 


52 


49 


49 


X 3 






— 


79 


76 


73 


80 


65 


52 


60 


•63 


57 


h 










75 


79 


80 


63 


54 


56 


59 


57 


h 










— 


70 


76 


61 


5.1 


54 


58 


56 


X 6 












— - 


78 


65 


54 


56 


56 


56 


X 7 














-— 


65 


63 


67 


66 


66 


X 8 


















47 


43 


44 


47 


X 9 




















56 


51 


54 


x io 






















59 


62 


X ll 
























55 


X 12 


























Nota: 


Decimal 


points 


have 


been 


omitted 


from 


corral 


ations. 
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Table 6 

IntercoTrelation Matrix for Rock II 





¥ 


x i 


X 2 X 3 


X 4 


X 5 


X 6 


X 7 


X S 


X 9 


X 10 


X ll 


X 12 


Y 




15 


-05 36 


19 


24 


26 


20 


09 


12 


31 


22 


16 


X l 






26 30 


23 


15 


19 


11- 




OS 


18 


13 


.0 


h 






— -05 


09 


10 


11 


-03 


=03 


01 


-02 


05 


63 


h 








56 


47 


40 


34 


13 


10 


51 


33 


22 


h 










50 


42 


24 


15 


12 


42 


24 


2 7 














35 


19 


01 


15 


28 


16 


20 


h 














21 


13 


09 


28 


28 


23 


h 
















13 


06 


29 


15 


14 


X S 


















06 


22 


38 


07 


X 9 




















10 


01 


12 


X 10 
\l 






















36 


22 
26 



Motel Decimal points have been omitted from correlat ions. 
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Population Validities of MR m& FRA Csrnpoiitcs for Reek. I 



Nfttnbef tit 
I'rciHrtorM 
R*,'»»rt'HH ton 


Average 
Papula feiws Vat IdlCy 
MR FRA 


Mean 


Standard 
Devise ion 
MR FRA 




N - 25 










2 


6S 


68 


03 


04 


02 


i 


63 


69 




06 


03 




60 


71 


u 


la 


03 




57 


71 


14 


1 3 


03 


b 


55 


71 


16 


15 


02 


7 


5S 


71 


16 


15 


02 


ft 


53 


71 


18 


15 


02 




53 


71 


18 


16 


01 


iU 


53 


71 


18 


15 


01 


n 


52 


71 


19 


15 


01 


i ^ 


52 


73 


19 


15 


01 














2 


70 


70 


00 


Ol 


01 


3 


6P 


70 


02 


04 


01 




67 


71 


04 


04 


01 


5 


67 


71 


06 


03 


01 


6 


66 


72 


m 


04 


01 


7 


66 


n 


01 


04 


01 


8 


65 


n 


01 


04 


01 


9 


64 


n 


OS 


04 


01 


10 


64 


m 


08 


04 


01 


11 


63 


71 


08 


05 


01 


12 


63 


71 


08 


05 


oo 




2 


70 


69 


-CI 


01 


01 




n 


70 


-Ol 


01 


01 




71 


71 


CO 


02 


01 




71 


72 


01 


02 


01 




n 


12 


01 


02 


01 




71 


11 


01 


02 


00 




71 


12 


01 


02 


00 


& 


71 


11 


01 


02 


00 


10 


71 


71 


01 


02 


00 


11 


71 


71 


©1 


02 


00 


12 


71 


71 


oo 


02 


00 


H * 1:00 


2 


70 


70 


oo 


01 


01 




72 


7\ 


-Ol 


01 


00 


4 


72 


72 


oo 


01 


01 


3 


72 


72 


oo 


01 


00 




73 


72 


•Ol 


00 


01 


7 


73 


72 


-01 


00 


01 


8 


73 


72 


^01 


01 


00 


9 


72 


72 


oo 


01 


00 


10 


72 


72 


oo 


01 


00 


r i 


73 


72 


-Ol 


01 


00 




73 


71 


-02 


01 


00 



Nut*; DeHmn! polniCn hjv* Net* ^silted. 
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Population ValldlClO of m and FRA Compoaic«s for lack 11 



Ml EJf f «lf i&* 


Jesulitiem Validity 

m wm 


Hun 


Standard 
Deviation 
m FRA 








• IS 










'2 


29 


34 


05 


10 


□ 7 




3 


30 


38 


08 


OS 


04 






31 


39 


08 


06 


03 






29 


40 


11 


□ 7 


01 




< 6 


29 


40 


11 


07 


02 




7 


27 


40 


13 


08 


02 




S 


27 


40 


13 


09 


02 




P 


27 


40 


1 1 


01 


02 




TiO 


27 


40 


13 


09 


02 






27 


40 


13 


Of 


02 






27 


40 


13 


09 


02 


- 




n 












a 


34 


36 


oa 


08 


07 




3 


34 


39 


05 


08 


05 




■A 


35 


39 


04 


□7 


02 




5 


35 


40 


05 


OR 
wo 


01 




& 


34 


40 


0§ 


09 


01 




7 


35 


4L 


06 


09 


01 




S 


34 


41 


07 


08 


01 




3 


35 


41 


wo 


08 


01 






34 


41 


07 


08 


01 




4- * 


34 


41 


07 


09 


01 






34 


41 


07 


01 


01 






N ™ 












.a 


36 


39 


03 


04 


02 




3 


36 


39 


03 


08 


01 




4 


36 


39 


03 


08 


02 




i 


37 


39 


02 


04 








37 


39 


02 


04 


00 






38 


39 


01 


04 


00 






36 


39 


01 


04 


00 






38 


39 


m 


04 


00 




4>v 


39 


3§ 


-01 


04 


01 






39 


38 


-01 


04 


00 




Id 


39 


37 


-02 


04 


00 






a m 












$ 


38 


39 


01 


02 


02 




13 


■: 37 


39 


02 


02 


02 




M 


38 


39 


01 


02 


02 






J? 


10 
J? 


00 


02 


01 






40 


39 


-01 


02 


01 






40 


39 


-01 


02 


00 






4Q 


38 


-02 


02 


00 






4Q . 


39 


-01 


02 


00 






40 


3S 


-02 


02 


01 






39 


37 


-02 


02 


00 






39 


38 


-01 


02 


00 
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Similar results are presented in Table 8 for the other population from 
Rock at; al* f (1970), FHA performed even better than in previous po pulat ions 
for small (two or three) predictor subsets* The average population validity 
for FRA with two predictors was higher than it was for MR in every sample 
size, Trua? the differences are small for the larger sample sizes. When 
N_ = 1QQ* however , there was much less fluctuation from s amp le- to-sample for 
FRA validities. When N = 200, the FRA validities were still more s table, in 
general* than the MR validities* 

The fourth population—like the second and third populations— a Iso had 
been analysed in a previous journal article — one devoted to comparing popula- 
tion validities of predictor selection techniques (McCornack, 1970) . Table 9 
contains the int er cor xelat ion matrix, and Table 10, the comparitive results* 
Since the$# tables include 15 predictors, the samples of 25 are dangerously 
close to total instability for multiple regression purposes, However s FRA 
performed very well, indeed, It gave a high and very stable population 
validity at: all subset sizes* Its average population validity urich 15 pre- 
dictors in regression was ,81—16 correlation points above the one for MS*, 
and its standard deviation was only *02 across the 10 samples. 

With sample sizes of 50^ Mil achieved high population validities, but they 
were less a. table than the FRA validities. Except for small subsets, MR and 
FRA performed! equally well in the larger samples* In the small subsets, e,g- , 
when only two or three predictors were in regression, FRA excelled, Even 
when C£ m 2O0 3 , FRA gave a higher and more stable population validity for the 
two-predictor subset, 

Population validities for unit weights were also computed for all four 
populations, Various studies have shown that unit weights of ten yield higher 
cross-validation than MR weights when small samples are used (e.g*, Perloffs 
1951, Note 3; Schmidt s 1971), With all predictors in regression, axid _N = 25, 
the population validities for FRA were larger than the population validities 
for unit weights", 07s »03, .03, and ,26 correlation points in the four popular 
tionss respectively* 

Additional analyses were also conducted in order to test the feasibility" 
and desirability of using the second principle component to enhance prediction, 
The mathematical procedure for combining the second component with the first 
component $a as to optimally predict Y was formulated* 1 Basically, it in- 
volved t\m computation of the best least square weighting vector Eox estimating 
the sample Y from the first two principle components. This proceduxe did not 
enhance prediction of Y in any of the populations, In fact , in general there 
was a slight attenuation (*01) of the prediction provided by the filist compoaerit 
alone* 



The author is indebted to Paul Horst tor his assistance. 
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Table 9 

Idtercorrelation Matrix from HcCornack 



H 





Y Xj ^2 Xj 


*5 


X 6 




X 8 


\ 


X 10 


X ll 


X 12 


X 13 








- 45 72 79 33 


-09 


-09 


02 


07 


-05 


1? 


OS 


0? 


-23 


-29 


-25 


x i 


- 52 35 H 


-09 


-01 


02 


-09 


0& 


oo 


-21 


04 


08 


11 


04 




66 11 


-U 


-01 


03 


-01 


-04 


13 


-01 


02 


-05 


-14 


-13 
























X 3 


- 30 


-03 


-01 


02 


02 


-05 


U 


06 


0? 


-20 


-18 


-17 














00 




-02 


00 


03 


-11 


04 


03 


-04 


03 


-01 


06 


x 4 

f 






















14 






— 


-43 


42 


33 


-10 


41 


03 


03 


-05 


-03 


A 5 
























-04 










00 


-01 


2? 


-I? 


-02 


OO 


01 


11 












-35 


-16 


. -26 


04 


-02 


-02 


03 


-16 


x 8 












-21 


-35 


0/ 


01 


-06 


-23 


•u 


x 9 
















-12 


05 


08 


23 


-10 


















10 


-02 


-11 


-33 


-16 


X il 


















49 


-42 


-38 


-12 


X l2 




















-35 


06 


04 


K 13 






















32 


10 


x 14 
























37 





























15 



Note! Decimal points have been omitted from correlations, 
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table 10 

FopvUtlon Validities of MR and FRA 
CttfepictBltea for MeCornaGk Data 



HxSvTor " ^ Average ~ ^ a " d ^ d 

PrtUirt4>fi Population Val Idle lea Mean Deviation 

in H^f.Mlnn FRA Difference MR FRA 

. — ~. * ^~™^^>S 80 02 ~~ 03 55T 

3 ?8 81 03 03 01 

4 78 81 03 04 Ol 
I 77 81 04 04 01 
b 76 81 05 06 01 
7 7 6 gi 05 07 02 
g 74 81 07 09 02 
9 74 SI 07 09 02 

l0 72 81 09 09 02 

u 7D 81 11 09 02 

12 69 81 12 09 02 

13 67 81 14 09 02 
u 68 81 13 09 02 



15 



65 81 16 10 02 



■N - 50 



2 


"" " SO 


83 


03 


03 


00 


3 


Si 


83 


02 


03 


00 


4 


82 


84 


02 


03 


01 


S 


82 


84 


02 


04 


01 


b 


S2 


84 


02 


04 


01 


7 


82 


84 


02 


04 


01 


8 


Si 


84 


03 


05 


01 


? 


Si 


84 


03 


06 


01 


10 

u 


SI 


84 


03 


06 


01 


Si 


84 


03 


06 


01 


12 


Si 


84 


03 


06 


01 


13 


81 


84 


03 


05 


01 


14 


91 


84 


03 


05 


01 


15 


Si 


84 


03 


06 


01 



I 


Si 


3 


S3 


4 


S3 


5 


S3 




84 


7 


84 


B 


84 


9 


84 


10 


84 


11 


84 


12 


as 


13 


85 


14 


as 


15 


84 



2 


80 


3 


83 


4 


84 


5 


83 


6 


83 


7 


84 


8 


84 


9 


84 


10 


84 


n 


84 


12 


84 


13 


84 


14 


84 


15 


84 



H - 100 



83 


02 


01 


00 


84 


01 


01 


00 


84 


01 


01 


00 


84 


01 


01 


00 


84 


00 


01 


01 


85 


01 


01 


01 


84 


00 


01 


01 


85 


01 


01 


01 


85 


01 


01 


01 


84 


00 


01 


01 


85 


00 


01 


01 


85 


00 


01 


01 


85 


01 


01 


00 


85 


01 


01 


00 



H m 200 



83 


03 


01 


00 


84 


01 


01 


00 


84 


00 


01 


uu 


84 


01 


01 


00 


84 


01 


01 


00 


84 


00 


01 


00 


84 


00 


01 


01 


84 


00 


01 


01 


84 


00 


01 


01 


84 


00 


01 


01 


84 


00 


01 


01 


85 


01 


01 


01 


85 


01 


01 


01 


85 


01 


01 


02 



Note I li^C-imy I jnilnlH have been amitted. 
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DISCUSSION AND CONCLUSIONS 



Factor Regression Analysis (FRA) went through a long developmental 
process* Thus, as presented above, it does not greatly resemble its earlier 
self. Nevertheless, judging from the reactions of colleagues to whom the 
author has presented FRA, its earlier forms and developments are very in- 
structive and meaningful . Some years ago the author had become convinced 
that the mathematical process called multiple regression (MR) was far too 
precise for most of our samples—that it assumed far more stability and 
reliability than was present in our data* Therefore! it capitalized on 
chance to such an extent that weights based on sample were likely to be 
very poor in the population from which the sample was drawn. It was felt 
chat some restraint could be placed on the mathematical process of weight 
determination by eliminating the actual Y scores while retaining the subjects 1 
order on the Y scale, This was done by pairing subjects adjacent on the Y 
scale—the highest with the next highest, the third highest with the fourth 
highest, etc • , until N/2 pairs were obtained, 

The author suspecu^d that these pairs could be used to compute "regres- 
sion coefficients 11 for predicting Y* without directly using the Y scores. 
Obviously, the Intrapair correlation on Y would be very high—it proved to 
be about . 99, even with rather small samples (e.g. , N_ = 50), (This is an 
intraclass correlation — both axes of the response surface refer to the same 
variable* and a plotted point represents one subject-pair.) 

What would the intrapair correlation be for a given predictor after 
matching on Y? The author felt this correlation would be, to some extent, 
dependent upon the predictor's validity, i.e., its correlation with Y. 
Conversations with other research psychologists and statisticians $ however , 
revealed that the dependency, if it existed, was certainly not obvious. 
Consequently, the empirical path was chosen temporarily* A small subsample 
from a large sample of Navy data was used, All intrapair correlations were 
computed. The correlations were generally much smaller than the respective 
validities j but were rather obviously correlated with them, So f the pos- 
sibility still remained that there was some dependency. 

Simultaneously, a computer program was written that would compute an 
intrapair correlation for all the predictors combined, weighting them so as 
to maximize this correlation, (One can conceive of this as an "intraclass 
canonical" correlation—a battery of variables is optimally weighted to cor- 
relate with itself, each plotted point of the response surface representing 
a subject pair,) These weights were then applied to the large sample from 
which the subsample was drawn* It was discovered that they predicted Y quite 
well— better than the multiple-regression weights from the same subsample. 

The computer program is based on Thomson's application (1940) of Hottelling's 
"most predictable criterion" .Mlution (1935), Thomson showed how one could 
obtain the composite that is maximally reliable by computing the first 
principle component of a correlation matrix with reliabilities in the diagonal* 
In the present case, the correlation matrix for the predictors was used, with 
each predictor's diagonal cell containing its intrapair correlation* The first 
principle component of this correlation matrix yielded the weights that proved 
to predict Y in the population so well, 
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Meanwhile, a search of the literature revealed an article by McNemar 
(1940) which indicated that the expected value of the intrapair eorrela- 
tlon for a given predictor might be its validity squared. A mathematical 
proof was developed and is presented in the appendix. Armed with this 
proof, it was no longer necessary to pair-off the subjects on the basis 
of their Y scores • The computer program was simply modified to use the 
squared validity instead of the intrapair correlation, in the respective 
diagonal ceil. Subsequent experience revealed that the validity itself 
worked as well, or even better for some populations, than the validity 
squared * 

Thus, the correlation matrix used in the computer program is the 
predictor matrix, with each predictor's validity in the diagonal cell for 
that predictor. Each and every element in the matrix is based on the entire 
sample of N subjects. The first-principle component of this matrix yields 
the FRA weights. This component has the greatest variance that is possible, 
representing the best-fitting line through the predictor space after the 
predictor scales have been transformed by their validities. Two predictors 
that are highly correlated can, and often do, have the same weight. This 
enhances the reliability of the composites. 

It may be of interest to compare FRA and MR in matrix^algebra form. 
Weight computation involves the solving of the Lagrange multiplier, A, 
through a set of simultaneous linear equations of the form! 

| r'R^r - X I j = 0 
in the case of MR, and 

|R =1 C - AX j - 0 

in the case of FRA, where 

r is the vector of validities , 

R is the inter cor relation matrix of predictors with 
ones in the main diagonal, 

I is the identity matrix, and 

C is the inter correlation matrix of predictors with 
validities in the main diagonal* 

The equation for FRA yields predictor weights that maximize the expected 
value of the intrapair correlation discussed above* 

It can be seen that the equations for MR and FRA differ only in the 
first term* In the case of MR, R-inverse is pre- and postmultiplied by the 
vector of validities. And in the case of FRA, it is post multiplied by C, 
which is the same as R except for the diagonal elements—unities In R, and 
validities in C, It is of interest to note that the FRA equation is equivale 
to the one used by Thomson (1940) to maximize the reliability of a composite, 
except that the diagonal elements of matrix C were the reliabilities of the 
respective predictors (see Gulliksen, 1950, p, 346), 
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For completeness * we should note that tp^ c°^g.raP^ equatiP* 1 for factor 
analysis is |C - Xl| ■ 0 where C is the int^^a 0 ^^ j at i°n matf^ *^th com- 
munali ties in the diagonal. When reliabilt^e^ u^ ec U as Often are 

as communality estimates, the C matrix is ty* a s the matrix u ^ed by 

Thomson (1940)* The remaining difference h^w^t\ jj ac t°t analyst ^tid maximiz- 
ing the reliability of a composite is that $ i s ^j^iflied by R ^inv er se 
in the latter case* 

The new method is a very important impt^e^it^ up^ n multiple r Agression 
for small samples, In addition, FRA does i^X at ^ e hitjhef s ^tiipi es 

sizes as previously proposed methods have cj^^e _ ^^ lllS f it can u s e d with 
confidence for all samples sizes as they oq^^t jX\ ^ppJ^d settin? 8 * 
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PROOF THAT THE VALIDITY SQUARED IS THE EXPECTED VALUE OF THE 
CORRELATION BETWEEN TWO SUBSETS MATCHED ON THE CRITERION 
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The sample members art* ranked on the basis of their criterion (Y) scores. 
Then, the sample member with the highest Y score is paired with the second 
highest, the third highest is paired with the fourth highest . etc* In this 
way s the sample is divided into two sets , A and B, which correlate as high 
as possible on the criterion, Y* Set A contains' all the odd rank numbers 
and set B contains all the even rank numbers* 

If we assume standard scores, the variance of the differences on a 
predictor, X, is 

"(A H)' = V 2 ' °B 2 ' 2r AB°AV (1) 

where r^g is the correlation between sets A and B on X s 

But since Ea A 2 - Eo G 2 = 1,0, 
A B 

K "(A-B) ? " 2 ^W' (2> 

Now, if we return to the original bivariate distribution X and Y for 
this sample, the standard error of estimate is 

°YX = "y^YX 7 ' (3) 



and for a given Y score 



°X-Y 2 ' 1 " r Yx''' (A) 



For the same Y score, the variance of the difference between the pair groups, 
sets A and B, is 

°(A=B) 2 = °A 2 + °B 2 + °» ' (5) 

where the zero arises from the fact that Er m 0 for a eiven Y score* 

AB ' 

But since, for a given Y score, 

Eo a " e "b " Ea r < 6) 



Equation 5 can be written 



or, following Equation 4, 



Ea (A-V 2 m 2(l-r 2). (8 ) 



A-l 



33 



LNotice that Equations 2 and 8 are both based on a conceptually fixed 
Y* In Equation 2, Y was fixed for each difference by matc^ng. In Equa- 
tion 8 S Y was fixed statistically, Therefore, Equations 2 and 8 are 
equivalent, and 



[aa- 



r _) 
A3 



Er 



AB 



^YX 



>. 



Thus, the validity squared is the expected value of the correlation between 
two subsets matched on the criterion, 
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